Papers with generation processes
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate draft tokens in large language models are expensive and resource-intensive. |
| Approach: | They propose an approach to generate draft tokens using a segment of the LLM and a self-distillation method to enhance the quality of draft token. |
| Outcome: | The proposed approach generates draft tokens using a segment of the LLM and a self-distillation method to improve quality and speed up generation. |
Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion (2025.naacl-long)
Copied to clipboard
Jacob K Christopher, Brian R. Bartoldson, Tal Ben-Nun, Michael Cardei, Bhavya Kailkhura, Ferdinando Fioretto
| Challenge: | Existing methods to accelerate large language model inference are limited by the reliance on incremental token generation in existing draft models. |
| Approach: | They propose an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences and allows parallelization of both the drafting and verification steps. |
| Outcome: | The proposed approach provides 7.2x speedups over standard generation processes and 1.75x speed ups over existing speculative decoding approaches. |
InferBR: A Natural Language Inference Dataset in Portuguese (2024.lrec-main)
Copied to clipboard
| Challenge: | Portuguese has few NLI-annotated datasets created through automatic translation followed by manual checking. |
| Approach: | They propose to generate premises and hypotheses using a semiautomatic process to generate sentences and manually check the annotations. |
| Outcome: | The proposed dataset is better at recognizing entailment classes in other Portuguese datasets than the reverse. |